An Auction-based Coordination Strategy for Task-Constrained Multi-Agent Stochastic Planning with Submodular Rewards

In many domains such as transportation and logistics, search and rescue, or cooperative surveillance, tasks are pending to be allocated with the consideration of possible execution uncertainties. Existing task coordination algorithms either ignore the stochastic process or suffer from the computational intensity. Taking advantage of the weakly coupled feature of the problem and the opportunity for coordination in advance, we propose a decentralized auction-based coordination strategy using a newly formulated score function which is generated by forming the problem into task-constrained Markov decision processes (MDPs). The proposed method guarantees convergence and at least 50% optimality in the premise of a submodular reward function. Furthermore, for the implementation on large-scale applications, an approximate variant of the proposed method, namely Deep Auction, is also suggested with the use of neural networks, which is evasive of the troublesome for constructing MDPs. Inspired by the well-known actor-critic architecture, two Transformers are used to map observations to action probabilities and cumulative rewards respectively. Finally, we demonstrate the performance of the two proposed approaches in the context of drone deliveries, where the stochastic planning for the drone league is cast into a stochastic price-collecting Vehicle Routing Problem (VRP) with time windows. Simulation results are compared with state-of-the-art methods in terms of solution quality, planning efficiency and scalability.Comment: 17 pages, 5 figure

Mission planning for a multiple-UAV patrol system in an obstructed airport environment

This paper investigates using multiple unmanned aerial vehicles (UAVs) to carry out routine patrolling at an airport to enhance its perimeter security. It specifically focuses on mission planning of the system to facilitate efficient patrolling with consideration of local buildings and restricted airspace. The proposed methodology includes three aspects: 1) a vision-based set cover algorithm to construct the patrolling network, 2) an obstructed partitioning-based clustering algorithm for recharging station placement, and 3) a mixture integer quadratic programming (MIQP) algorithm to plan routes for UAVs minimizing the maximum idle time through out all patrolling waypoints. The main contribution of this work is that it provides a comprehensive mission planning solution for UAVs persistently patrolling in a complex environment characterized by blocked vision and restricted airspace. The proposed methodology is evaluated through intensive simulations in the context of the Cranfield Airport scenario.Innovate UK: 1002481

Distributed optimal deployment on a circle for cooperative encirclement of autonomous mobile multi-agents

A distributed encirclement points deployment scheme for a group of autonomous mobile agents is addressed in this paper. Herein, each agent can measure its own azimuth related to the common target and can at least communicate with its two adjacent neighbors. Given its space-cooperative character, the encirclement points deployment problem is formulated as the coverage control problem on a circle. The measurement range of azimuth sensor is taken into consideration when doing problem formulation, which is closer to the facts in real-world applications. Then, the fully distributed control protocols are put forward based on geometric principle and the convergence is proved strictly with algebraic method. The proposed control protocols can steer the mobile agents to distribute evenly on the circle such that the coverage cost function is minimized, and meanwhile the mobile agents' spatial order on the circle is preserved throughout the systems' evolution. A noteworthy feature of the proposed control protocols is that only the azimuths of a mobile agent and its two adjacent neighbors are needed to calculate the mobile agent's control input, so that the control protocols can be easily implemented in general. Moreover, an adjustable feedback gain is introduced, and it can be employed to improve the convergence rate effectively. Finally, numerical simulations are carried out to verify the effectiveness of the proposed distributed control protocol

Edge-enhanced attentions for drone delivery in presence of winds and recharging stations

Existing variants of vehicle routing problems have limited capabilities in describing real-world drone delivery scenarios in terms of drone physical restrictions, mission constraints, and stochastic operating environments. To that end, this paper proposes a specific drone delivery problem with recharging (DDP-R) characterized by directional edges and stochastic edge costs subject to wind conditions. To address it, the DDP-R is cast into a Markov decision process over a graph, with the next node chosen according to a stochastic policy based on the evolving observation. An edge-enhanced attention model (AM-E) is then suggested to map the optimal policy via the deep reinforcement learning (DRL) approach. The AM-E comprises a succession of edge-enhanced dot-product attention layers and is designed with the aim of capturing the heterogeneous node relationship for DDP-Rs by incorporating adjacent edge information. Simulations show that edge enhancement facilitates the training process, achieving superior performance with less trainable parameters and simpler architecture in comparison with other deep learning models. Furthermore, a stochastic drone energy cost model in consideration of winds is incorporated into validation simulations, which provides a practical insight into drone delivery problems. In terms of both nonwind and windy cases, extensive simulations demonstrate that the proposed DRL method outperforms state-of-the-art heuristics for solving DDP-Rs, especially at large sizes

Distributed target-encirclement guidance law for cooperative attack of multiple missiles

The target-encirclement guidance problem for many-to-one missile-target engagement scenario is studied, where the missiles evenly distribute on a target-centered circle during the homing guidance. The proposed distributed target-encirclement guidance law can achieve simultaneous attack of multiple missiles in different line-of-sight directions. Firstly, the decentralization protocols of desired line-of-sight angles are constructed based on the information of neighboring missiles. Secondly, a biased proportional navigation guidance law that can arbitrarily designate the impact angle is cited. The missiles can achieve all-aspect attack on the target in an encirclement manner by combining the biased proportional navigation guidance law and dynamic virtual targets strategy. Thirdly, the consensus protocol of simultaneous attack is designed, which can guarantee that all missiles’ time-to-go estimates achieve consensus asymptotically, and the convergence of the closed-loop system is proved strictly via the Lyapunov stability theory. Finally, numerical simulation results demonstrate the performance and feasibility of the proposed distributed target-encirclement guidance law in different engagement situations

Decentralized task allocation for multiple UAVs with task execution uncertainties

This work builds on a robust decentralized task allocation algorithm to address the multiple unmanned aerial vehicle (UAV) surveillance problem under task duration uncertainties. Considering the existing robust task allocation algorithm is computationally intensive and also has no optimality guarantees, this paper proposes a new robust task assignment formulation that reduces the calculation of robust scores and provides a certain theoretical guarantee of optimality. In the proposed method, the Markov model is introduced to describe the impact of uncertain parameters on task rewards and the expected score function is reformulated as the utility function of the states in the Markov model. Through providing the high-precision expected marginal gain of tasks, the task assignment gains a better accumulative score than the state of arts robust algorithms do. Besides, this algorithm is proven to be convergent and could reach a prior optimality guarantee of at least 50%. Numerical Simulations demonstrate the performance improvement of the proposed method compared with basic CBBA, robust extension to CBBA and cost-benefit greedy algorithm

The Subsea Micro-Drilling Vehicle’s Dynamic Analysis during Landing

Whether the subsea micro-drilling vehicle (SMDV) can perform its subsequent operations safely depends on the quality of the landing procedure. RecurDyn creates the SMDV dynamic model for this study. A model of the interaction between the SMDV and deep-sea sediment is built, and a simulation of the SMDV falling on the sea’s sediment substrate is developed. The water resistance is applied to the model by equivalent height replacement, and the in-situ soil data is measured with a triaxial undrained unconsolidated (UU) compression test and a load-sinkage experiment. When the landing surface is a flat sediment substrate, the release height is 5 m, the sinkage amount is 347 mm, and the center of mass’s impact acceleration is less than seven gravitational accelerations. Three states can occur when the vehicle lands on a sloped surface: stability, slip, and overturning. The risk of slipping and overturning is the least when the vehicle is landing on the ground in the forward direction, and the risk is equal when it lands on the ground in the backward and sideways directions. The ultimate overturning angle drops, and the final slip angle remains relatively constant as the vehicle’s release height increases. Our findings offer a theoretical foundation for the SMDV’s safe landing and the scientific formulation of rational release intervals

Constitutive modeling for the flow stress behaviors of alloys based on variable order fractional derivatives

During hot working, alloys may experience three kinds of flow stress behaviors, including strain hardening, strain softening, or steady flow, because of the competition of work hardening and thermal softening. Modelling the flow stress behaviors plays an essential role in understanding the mechanical properties of alloys. In this paper, the variable order fractional model is provided to describe the flow stress behaviors of alloys. The variation of the fractional order between 0 and 1 can reflect the mechanical property changing between solids and fluids. By assuming that the fractional order varies linearly with time, the proposed model can describe both the strain softening and strain hardening behaviors of alloys. The model fitting results are compared to the experimental data of A356 alloy for strain softening and Cu-Cr-Mg alloy for strain hardening under different temperatures and strain rates. It is validated that the variable order fractional model can accurately describe the flow stress behaviors of alloys. Furthermore, the rule of the variable order is also discussed to analyze its overall values and the changes before and after the yield point. It is concluded that the variation of the fractional order can intuitively reveal the changes in mechanical properties in the flow stress behaviors of alloys, including both strain softening and strain hardening

Delivery route planning for unmanned aerial system in presence of recharging stations

Existing variants of vehicle routing problems (VRPs) are incapable of describing real-world drone delivery scenarios in terms of drone physical restrictions, mission constraints, and stochastic operating environments. To that end, this paper proposes a specific drone delivery model with recharging (DDP-R) characterized by directional edges and stochastic edge costs subject to wind conditions. To address it, the DDP-R is cast into a Markov Decision Process (MDP) over a graph, with the next node chosen according to a stochastic policy based on the evolving observation. An edge-enhanced attention model (AM-E) is then suggested to map the optimal policy via the deep reinforcement learning (DRL) approach. AM-E comprises a succession of edge-enhanced dot-product attention layers which is designed with the aim of capturing the heterogeneous node relationship for DDP-Rs by incorporating adjacent edge information. Simulation shows that edge enhancement facilitates the training process, achieving superior performance with less trainable parameters and simpler architecture in comparison with other deep learning models. Furthermore, a stochastic drone energy cost model in consideration of winds is incorporated into validation simulations, which provides a practical insight into drone delivery problems. In terms of both non-wind and windy cases, extensive simulations demonstrate that the proposed DRL method outperforms state-of-the-art heuristics for solving DDP-R, especially at large sizes